Sampling grid information for spatial layers in multi-layer video coding

ABSTRACT

Sampling grid information may be determined for multi-layer video coding systems. The sampling grid information may be used to align the video layers of a coding system. Sampling grid correction may be performed based on the sampling grid information. The sampling grids may also be detected. In some embodiments, a sampling grid precision may also be detected and/or signaled.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/680,244, filed Aug. 6, 2012, titled “Signaling OfSampling Grid Information Among Spatial Layers In Multi-Layer Video”,and U.S. Provisional Patent Application No. 61/809,102, filed Apr. 5,2013, titled “Signaling Of Sampling Grid Information Among SpatialLayers In Multi-Layer Video Coding”, the disclosures of bothapplications being herby incorporated by reference herein in theirrespective entirety, for all purposes.

BACKGROUND

As video applications are being implemented on smartphones and otherWTRUs, video consumption may be accommodated on devices with widelyvarying capabilities (e.g., in terms of computing power, memory/storagesize, display resolution, display frame rate, etc.). The network and/ortransmission channels may also have widely varying characteristics(e.g., in terms of packet loss rate, available channel bandwidth, bursterror rate, etc.). Moreover, the video data may be transmitted overwired networks and/or wireless networks. The various configurations ofthese devices and/or network entities may complicate the transmissionand/or coding of the video data.

SUMMARY

Systems, methods, and apparatuses are described herein for signalingsampling grid information in a multi-layer video coding system. Forexample, sampling grid information may be received that may indicate atleast a degree (or amount) of sampling grid alignment between a firstvideo layer and a second video layer. A sampling grid associated withthe first video layer may be offset according to the received samplinggrid information to align with a sampling grid associated with thesecond video layer.

In one or more embodiments, sampling grid correction may be performedbased on sampling grid information.

In one or more embodiments, the sampling grids may be detected.

Embodiments contemplate one or more techniques for sampling gridcorrection for multi-layer video coding. Techniques may includereceiving sampling grid information that may indicate a sampling gridalignment between a first video layer and a second video layer.Techniques may also include aligning a sampling grid associated with thefirst video layer with a sampling grid associated with the second videolayer. The aligning may be based on the received sampling gridinformation.

Embodiments contemplate one or more techniques for communicatingsampling grid information for multi-layer video coding. Techniques mayinclude receiving a first data (e.g., a data set or data). Techniquesmay also include determining if the first data includes the samplinggrid information. Techniques may also include determining a precision ofthe sampling grid information upon the first data being determined toinclude the sampling grid information.

Embodiments contemplate one or more techniques for detection of arelative sampling grid for one or more layers in multi-layer videocoding. Techniques may include upsampling a base layer to match anenhancement layer spatial resolution. Techniques may also includeapplying adaptive filter training to the upsampled base layer and theenhancement layer. Techniques may also include determining a position ofthe relative sampling grid based on one or more adaptive filtercoefficients from the adaptive filter training.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an example block-based hybrid scalablevideo encoding system with N layers, consistent with embodiments.

FIG. 2 shows a diagram of an example of temporal and inter-layerprediction for stereoscopic (2-view) video coding using H.264/MVC,consistent with embodiments.

FIG. 3 shows a block diagram of an example spatial scalable multi-layer(e.g., 2 layer) encoding system, consistent with embodiments.

FIG. 4 shows a block diagram of an example spatial scalable multi-layer(e.g., 2 layer) decoder, consistent with embodiments.

FIGS. 5A (e.g., zero phase shift) and 5B (e.g., non-zero phase shift)show examples of a sampling grid between a BL video and an EL video fora down-sampling ratio (e.g., 2×, each dimension), consistent withembodiments.

FIGS. 6A (e.g., zero phase shift) and 6B (e.g., non-zero phase shift)show other examples of a sampling grid between a BL video and an ELvideo for a down-sampling spatial ratio (e.g., 1.5×, each dimension),consistent with embodiments.

FIG. 7 shows an example of a misaligned (or mismatched) sampling gridbetween an EL video and an up-sampled BL video after down-sampling andup-sampling (e.g., 2× spatial ratio, non-zero phase shift down-samplingand zero phase shift up-sampling), consistent with embodiments.

FIG. 8 shows an example flow diagram for signaling of sampling gridinformation (e.g., video parameter set(s) (VPS) and/or sequenceparameter set(s) (SPS)), consistent with embodiments.

FIG. 9 shows an example for signaling sampling grid information,consistent with embodiments.

FIG. 10 shows a block diagram for an example detection of a samplinggrid (e.g., relative sampling) between spatial layers (e.g., two spatiallayers), consistent with embodiments.

FIG. 11 shows a block diagram that illustrates an example relativesampling grid detection using 2-tap adaptive filter (e.g., X dimension),consistent with embodiments.

FIG. 12 shows a block diagram of an example multi-layer (e.g., 2 layer)spatial scalable encoding system including sampling grid informationsignaling, sampling grid correction, and sampling grid detection,consistent with embodiments.

FIG. 13A is a system diagram of an example communications system inwhich one or more disclosed embodiments may be implemented.

FIG. 13B is a system diagram of an example wireless transmit/receiveunit (WTRU) that may be used within the communications systemillustrated in FIG. 13A.

FIG. 13C is a system diagram of an example radio access network and anexample core network that may be used within the communications systemillustrated in FIG. 13A.

FIG. 13D is a system diagram of another example radio access network andan example core network that may be used within the communicationssystem illustrated in FIG. 13A.

FIG. 13E is a system diagram of another example radio access network andan example core network that may be used within the communicationssystem illustrated in FIG. 13A.

DETAILED DESCRIPTION

A detailed description of illustrative embodiments will now be describedwith reference to the various Figures. Although this descriptionprovides a detailed example of possible implementations, it should benoted that the details are intended to be exemplary and in no way limitthe scope of the application. As used herein, the article “a” or “an”,absent further qualification or characterization, may be understood tomean “one or more” or “at least one”, for example.

Various digital video compression technologies may be used to enableefficient digital video communication, distribution, and/or consumption.Examples of commercially deployed standards for such digital video maybe developed by ISO/IEC and ITU-T, such as H.26, MPEG-1, MPEG-2, H.263,MPEG-4 part 2, H.264/MPEG-4 part 10 AVC, and/or High Efficiency VideoCoding (HEVC) (e.g., which may be jointly developed by ITU-T VideoCoding Experts Group (VCEG) and ISO/IEC Moving Picture Experts Group(MPEG)). HEVC may be used to achieve greater compression (e.g., twice asmuch compression as H.264/AVC), and/or less bit rate (e.g., half the bitrate) in the same, or similar, video quality.

Digital video services may be implemented via TV services oversatellite, cable, terrestrial broadcasting channels, the internet (e.g.,on mobile devices, such as smart phones, tablets, and/or other WTRUs)and/or other video applications, such as video chat, mobile videorecording and sharing, and/or video streaming for example. These videotransmissions may be performed in heterogeneous environments. In varioustypes of video transmission scenarios, such as 3-screen or N-screen,various consumer devices (e.g., PCs, smart phones, tablets, TVs, etc.)may accommodate video consumption with widely varying capabilities(e.g., in terms of computing power, memory/storage size, displayresolution, display frame rate, etc.). The network and/or transmissionchannels may have widely varying characteristics (e.g., in terms ofpacket loss rate, available channel bandwidth, burst error rate, etc.).Video data may be transmitted over a combination of wired networksand/or wireless networks, which may complicate the underlyingtransmission channel characteristics. In such scenarios, among others,the premise of scalable video coding (SVC) may be used to improve thequality of experience for video applications running on devices withdifferent capabilities over heterogeneous networks.

SVC may be used to encode the signal (e.g., in some embodiments perhapsonce) at a highest representation (e.g., temporal resolution, spatialresolution, quality, etc.). SVC may enable decoding from subsets of thevideo streams depending on the rate and/or representation implemented byapplications running on a client device. SVC may save bandwidth and/orstorage compared to non-scalable solutions. The international videostandards MPEG-2 Video, H.263, MPEG4 Visual and H.264 may have toolsand/or profiles that may support at least some modes of scalability.

FIG. 1 shows a block diagram of an example block-based hybrid scalablevideo encoding system. As illustrated in FIG. 1, the scalable videoencoding system may include a number of layers (e.g., 1-N layers). Thespatial/temporal signal resolution to be represented by the layer 1(e.g., the base layer) may be generated by down-sampling of the inputvideo signal. In a subsequent encoding stage, an appropriate setting ofthe quantizer (Q₁) may lead to a quality level of the base information.To encode (e.g., efficiently encode) the subsequent higher layers, thebase-layer reconstruction Y₁, an approximation of the higher layerresolution levels, may be utilized in the encoding/decoding of thesubsequent layers. The base layer reconstruction Y₁ may be upsampled tolayer-2's resolution, for example by an up-sampling unit. Down-samplingand/or up-sampling may be performed throughout the various layers (e.g.,layers 1, 2 . . . N). The down-sampling and/or up-sampling ratios may bedifferent, perhaps in some embodiments depending on the relativedimensions between the two given layers (e.g., n1 and n2), among otherfactors. The video signals represented by the two layers may have thesame, or similar, spatial resolution. In some embodiments thecorresponding down-sampling and/or up-sampling operations may beby-passed.

In the example scalable system shown in FIG. 1, for any given higherlayer n (e.g., 2≤n≤N), a differential signal may be generated bysubtracting an up-sampled lower layer signal (e.g., layer n−1 signal)from the current layer n signal, and the difference signal thus obtainedmay be encoded. In some embodiments, coding of a difference signal asimplemented in FIG. 1 may make it difficult to achieve good codingefficiency and/or avoid potential visual artifacts at the same time. Tonormalize the difference signal's dynamic range, non-linear quantizationand/or joint quantization and normalization may be used. In someembodiments, these processes may introduce visual artifacts that may bedifficult to avoid. In some embodiments, scalable video coding (e.g., asmay be implemented in video coding standards, such as the Scalable VideoCoding (SVC) and/or the Multi-View Coding (MVC) of the H.264 standard)might not use residual based inter-layer prediction. Instead, in someembodiments, inter-layer prediction based directly on decoded picturesof other layers may be used.

FIG. 2 shows an example of temporal and inter-layer prediction forstereoscopic (2-view) video coding using a left view (layer 1) and aright view (layer 2). As illustrated in FIG. 2, an example structure mayuse MVC to code a stereoscopic video with left view (layer 1) and rightview (layer 2). The left view video may be coded with IBBP predictionstructure, while the right view video may be coded with PBBB predictionstructure. In the right view (layer 2), the first picture collocatedwith the first I picture in the left view (layer 1) may be coded as a Ppicture, perhaps for example based on predicting from the I picture inlayer 1. The other pictures in the right view may be coded as Bpictures, with the first prediction coming from temporal references inthe same layer (e.g., the right view), and/or the second predictioncoming from inter-layer reference pictures in the layer 1 (e.g., theleft view), for example.

While FIG. 2 shows an example of view scalability, a scalable system maysupport other types of scalabilities. Table 1 illustrates several typesof scalabilities and example standards that may support them. Describedherein are one or more embodiments for spatial scalability (e.g., wherethe spatial resolutions of two layers may be different). One or moreembodiments are also described herein for the coding efficiency of ascalable system that may incorporate spatial scalability.

TABLE 1 Examples of different types of scalabilities Scalability ExampleStandards View scalability 2D→3D (2 or more MVC, MFC, 3DV views) Spatialscalability 720p→1080p SVC, scalable HEVC Quality (SNR) 35 dB→38 dB SVC,scalable HEVC scalability Temporal scalability 30fps→60fps H.264/AVC,SVC, scalable HEVC Standards scalability H.264/AVC→HEVC 3DV, scalableHEVC Bit-depth scalability 8-bit video → 10-bit Scalable HEVC* videoChroma format YUV4:2:0→YUV4:2:2, Scalable HEVC* scalability YUV4:4:4Aspect ratio scalability 4:3→16:9 Scalable HEVC* *might not be plannedin the first phase of scalable HEVC standardization

FIG. 3 shows an example 2-layer encoding system that may support spatialscalability. As shown, the base layer (BL) video input and theenhancement layer (EL) video input may be correlated with adown-sampling process. The BL video may be encoded by the BL encoder.The EL video may be encoded by the EL encoder. The reconstructed BLvideo signal, which may be stored in the BL reference picture buffer forexample, may be up-sampled to match the EL video resolution and/or usedby the EL encoder to efficiently encode and/or predict the EL videoinput. The up-sampling process illustrated in FIG. 3 may be applied atthe picture level (e.g., the entire BL picture may be up-sampled), atthe region level (e.g., portions of the BL picture may be up-sampled),and/or at the block level (e.g., some blocks in the BL picture may beup-sampled). After encoding, the video packets of the BL bit-streamand/or the EL bit-stream may be multiplexed together to form thescalable bit-stream. FIG. 4 shows an example spatial scalable decoder(e.g., with 2 layers) that in some embodiment may correspond to theencoder in FIG. 3.

In some embodiments, a scalable coding system may mandate theup-sampling process, e.g., common to the encoder in FIG. 3 and/or thedecoder in FIG. 4, perhaps to ensure bit exactness between the encoderand decoder, among other reasons. The down-sampling process in FIG. 3may be a pre-processing step and/or might not be mandated by anyscalable coding standard. The encoder may be allowed the freedom todetermine the frequency characteristics and/or the relative samplinggrid when designing the down-sampling filters that may be used indown-sampling. As described herein, in some embodiments, the encoder maychoose to apply a down-sampling process that, perhaps when combined withthe up-sampling process that may be mandated by the scalable codingsystem, might not produce a sampling grid that is fully aligned with theEL video signal. Consequently, the scalable coding performance may benegatively impacted. One or more embodiments are described herein forrecovering the sampling grid of the EL video. For example, the samplinggrid of the EL video may be recovered by signaling the sampling gridinformation as part of the scalable bit-stream and/or aligning thesampling grids between the spatial layers, perhaps based on the samplinggrid information, among other factors.

FIGS. 5A, 5B, 6A, and 6B illustrate examples of sampling grids betweenthe BL video (e.g., circle) and the EL video (e.g., empty square) fortwo down-sampling ratios, 2× and 1.5×. For each spatial ratio, twoexemplar sampling grids are given, one with zero phase shift (in FIG. 5Aand FIG. 6A), and the other with non-zero phase shift (FIG. 5B and FIG.6B). The up-sampling filter may ensure bit exactness between the encoderand the decoder. The specified up-sampling filter may assume a fixedsampling grid between the two spatial layers during down-sampling. Thedevelopment of the scalable extension (e.g., of the HEVC standard) mayuse test video sequences generated using zero phase shift down-samplingfilters. As a result, the up-sampling filters (e.g., specified in thescalable extension of HEVC) may be zero phase shift up-sampling filtersso that the up-sampled BL video may be phase aligned with the EL video.

Encoder applications may choose to apply different down-samplingfilters. For example, they may choose to apply down-sampling filterswith non-zero phase shifts. Using an example of 2× spatial ratio betweenthe BL and the EL video, a down-sampling filter with non-zero phaseshift, and an up-sampling filter with zero phase shift, FIG. 7 gives anexample of a misaligned sampling grid between the EL video and theup-sampled BL video. In FIG. 7, the original EL video is shown in emptysquares. Perhaps after a non-zero phase shift down-sampling filter maybe applied, among other techniques, a sampling grid of the down-sampledBL video is shown in circles. Perhaps after a zero phase shiftup-sampling filter may be applied to the BL video, among othertechniques, the sampling grid of the up-sampled video is shown inpatterned squares. In some embodiments, perhaps when the up-sampledvideo that exhibits such misalignment in sampling grids may be used topredict the EL video, according to FIG. 3 and/or FIG. 4 for example(among other techniques), coding efficiency of the enhancement layervideo in terms of rate-distortion performance may be negativelyimpacted. Embodiments contemplate that perhaps while motion estimationwith fractional pixel precision may be used for compensation, the impacton coding performance might not be fully addressed due to one or more ofthe reasons described herein.

For example, some blocks in the EL video may be predicted using intraprediction from base layer, such as using the base layer intra skip modein H.264/SVC for example, where motion vectors might not be sent. Thevideo coding may support up to one-fourth pixel motion vector precisionfor one or more luma components and/or corresponding precision for oneor more chroma components, for example. In some embodiments, perhaps ifthe sampling grids may be misaligned by other finer fractions (e.g.,misaligned by one-eighth pixels), among other reasons, fractional motioncompensation might not be used. The video coding may support motionvector prediction, where block motion vectors may be predicted fromtheir neighboring blocks. In some embodiments, perhaps if a block in theEL video may be predicted from up-sampled BL video and/or may sufferfrom the misaligned sampling grid problem, and/or its neighbors may bepredicted from temporal reference and/or might not suffer from themisaligned sampling grid problem, or vice versa, among other reasons,motion vector prediction accuracy may be reduced.

One or more embodiments described herein may be implemented to signalsampling grid information. For example, the amount (or degree) ofsampling grid misalignment may be signaled. Corrections may be made tothe misaligned sampling grids, perhaps based on the signaled samplinggrid information, among other factors, for example.

The encoder may choose the down-sampling filters. The up-samplingfilters in the scalable system may be predetermined and/or fixed. Thedown-sampling filters may be chosen and/or fixed at a high level, e.g.,at the sequence level or above, perhaps to ensure that the pictures inthe same layer may share the same sampling grid (e.g., otherwise eventemporal prediction performance may be negatively impacted), among otherreasons. Syntax elements related to the sampling grid information may besent at the sequence level, such as but not limited to in the VideoParameter Sets (VPS) and/or Sequence Parameter Sets (SPS), for example.

FIG. 8 shows a flow chart for an example signaling of sampling gridinformation. As illustrated in FIG. 8, a flag may be sent to indicatewhether sampling grid information is present. In some embodiments,perhaps if such information might not be present, among other reasons, adefault sampling grid may be assumed. For example, the default mayindicate that the sampling grids of the up-sampled base layer and/or theenhancement layer may be aligned (e.g., zero phase shift between the twogrids). In some embodiments, perhaps if the flag may indicate thatsampling grid information is present, among other reasons, the precisionof the sampling grid information may be signaled, which may be followedby the actual sampling grid alignment information for the X and/or the Ydimension (e.g., a sampling grid offset in the X direction (or a firstdirection, such as but not limited to a horizontal direction) and/or asampling grid offset in the Y direction (or a second direction, such asbut not limited to a vertical direction)). In some embodiments, thetechniques may be applied to a data set (e.g., a set of data or data)that may include a VPS and/or SPS, for example. None of the individualelements described in regard to the flowchart in FIG. 8 are eithernecessary or essential, and any of the elements may be performed in anyorder, or in any combination.

In the example of FIG. 7, the sampling grid alignment (and/or amountand/or degree of alignment) between the up-sampled base layer and theenhancement layer may be signaled in one-fourth pixel precision (whichin some embodiments may by measured in terms of an enhancement layer(EL) sampling grid), for example, (among other contemplated precisions,including other fractional pixel precisions). In some embodiments, avalue (e.g., “+2”) may be signaled for the offset in one or more, oreach, dimension (e.g., to indicate that the relative sampling grids areoff by one-half pixels to the right and/or by one-half pixels down).Though not shown in the example of FIG. 7, the relative sampling gridsbetween the up-sampled BL and the EL may be off in one or bothdirections (e.g., a first direction and/or a second direction). That is,relative to the EL grid, the up-sampled BL grid may be to the left or tothe right in the X dimension, and/or above or below in the Y dimension.In some embodiments, the X and Y sampling grid offsets in FIG. 8 may besigned values, unsigned integer values, or a combination of both signedand unsigned integer values. The relative sampling grid between theup-sampled BL and the EL video may be off by less than ±1 full pixel,for example. In some embodiments, the offset precision in a firstdimension (e.g., X dimension) and in a second dimension (e.g., Ydimension) may be the same or substantially similar. In someembodiments, the offset precision in the X dimension and in the Ydimension may be different. In one or more embodiments, the offsetprecision in the X dimension and/or in the Y dimension may be signaledtogether or separately.

Sampling grid information may be signaled (e.g., in a video or on adisplay). In the examples of FIGS. 5A, 5B, 6A, and 6B, the BL and ELsampling grids are overlaid, for example to illustrate how the relativesampling grid information may be signaled. The precision may be measuredin terms of a BL sampling grid. As shown in FIG. 9, the distance of twoneighboring BL pixels may be divided (e.g., evenly) into 16 segments.One or more, or each, segment may represent 1/16-pixel unit ofprecision, for example. Other precisions may be implemented. Therelative positions of the pixels in the BL and EL pictures may beexamined and the sampling of grid offsets may be determined. Therelative positions that may be examined may include the upper-leftpixels in BL and EL pictures. The sampling grid offset in the horizontaldirection may be determined using Equation (1). The sampling grid offsetin the vertical direction may be determined using Equation (2).sampling_grid_offset_hori=position_BL_hori−position_EL_hori  Equation(1)sampling_grid_offset_vert=position_BL_vert−position_EL_vert  Equation(2)

The position_BL_hori and position_EL_hori may include the horizontalpositions of the upper-left pixels in the BL and EL pictures,respectively. The position_BL_ vert and position_EL_vert may include thevertical positions of the upper-left pixels in the BL and EL pictures,respectively. In FIG. 9, the sampling grid offsetssampling_grid_offset_hori and sampling_grid_offset_vert may be equal to4, with 1/16-pixel precision. In some embodiments, perhaps if the BL andEL sampling grids may be aligned, as shown in FIG. 5A and FIG. 6A, forexample, among other scenarios, the sampling grid offsets may includethe default values of zero.

The sampling grid offset may be determined for luma and/or chromaplanes. A pair of sampling grid offsets (e.g., sampling_grid_offset_horiand sampling_grid_offset_vert) may be signaled for one or more, or each,of the luma and/or chroma planes. The relative grids of the luma andchroma planes in a video sequence may be predetermined and/or fixed. Therelative grids may be fixed since the video sequence may include thestandardized color subsampling format, such as 4:2:0, 4:2:2, and/or4:4:4 for example. The sampling grid offsets for chroma planes may bederived according to the sampling grid offsets for the luma plane. Insome embodiments, perhaps when the sampling grid offsets for chromaplanes may be derived according to the sampling grid offsets for theluma plane, among other reasons, the sampling grid offsets for thechroma planes may be determined without explicit signaling. Thederivation process may be performed as described herein.

TABLE 2 Example of signaling sampling grid informationsampling_grid_information( ) { Descriptor sampling_grid_info_present_flag u(1)  if (sampling_grid_info_present_flag ) {   log2_sampling_grid_precision ue(v)  sampling_grid_abs_offset_x u(v)   sampling_grid_abs_offset_y u(v)  sampling_grid_sign_x u(1)   sampling_grid_sign_y u(1)  chroma_sampling_grid_info_presented_flag u(1)   if(chroma_sampling_grid_info_presented_flag ) {   log2_chroma_sampling_grid_precision ue(v)   chroma_sampling_grid_abs_offset_x u(v)   chroma_sampling_grid_abs_offset_y u(v)    chroma_sampling_grid_sign_xu(1)    chroma_sampling_grid_sign_y u(1)   }  }

Table 2 illustrates an example syntax table for signaling the samplinggrid information (e.g., in the segment of syntax elements illustratedherein). One or more of the semantics are further described herein. Thesampling_grid_info_presented_flag may be equal to 1 and/or may indicatethat sampling grid information can be (or in some embodiments, perhapswill be) presented in the following bitstream. Thesampling_grid_info_presented_flag may be equal to 0 and/or may indicatethat sampling grid information might not (or in some embodiments,perhaps will not) be presented. The sampling_grid_info_presented_flagmay indicate that the BL and EL sampling grids are aligned and/or thatthe offset is set to the default value of zero, perhaps when thesampling_grid_info_presented_flag may indicates that the sampling gridinformation might not be (or in some embodiments perhaps will not be)presented, for example. The log 2_sampling_grid_precision may indicatethe precision of the sampling grid. For example, the precision of thesampling grid may be determined by SamplingGridPrecision=2^(log 2) ^(_)^(sampling) ^(_) ^(grid) ^(_) ^(precision).

The sampling_grid_abs_offset_x may indicate the absolute value of thesampling grid offset in the horizontal direction. Thesampling_grid_abs_offset_x may be fixed-length coded. The code lengthmay be equal to log 2_sampling_grid_precision.

The sampling_grid_abs_offset_y may indicate the absolute value of thesampling grid offset in the vertical direction. Thesampling_grid_abs_offset_y may be fixed-length coded. The code lengthmay be equal to log 2_sampling_grid_precision.

The sampling_grid_sign_x may indicate the sampling grid offset in thehorizontal direction. The sampling_grid_sign_x being equal to 0 mayindicate that the sampling grid offset in the horizontal direction ispositive. The sampling_grid_sign_x being equal to 1 may indicate thatthe sampling grid offset in the horizontal direction is negative.

The sampling_grid_sign_y may indicate the sampling grid offset in thevertical direction. The sampling_grid_sign_y being equal to 0 mayindicate that the sampling grid offset in the vertical direction ispositive. The sampling_grid_sign_y being equal to 1 may indicate thatthe sampling grid offset in the vertical direction is negative.

The sampling grid offset in the horizontal direction may be indicated bythe SamplingGridOffsetX. The SamplingGridOffsetX may be determined bySamplingGridOffsetX=(1−sampling_grid_sign_x*2)*sampling_grid_abs_offset_x.The sampling grid offset in the vertical direction may be indicated bythe SamplingGridOffsetY. The SamplingGridOffsetY may be determined bySamplingGridOffsetY=(1−sampling_grid_sign_y*2)*sampling_grid_abs_offset_y.

Chroma sampling grid information may be indicated bychroma_sampling_grid_info_presented_flag. Thechroma_sampling_grid_info_presented_flag being equal to 1 may indicatethat chroma_sampling grid information may be (or in some embodimentsperhaps will be) presented in the following bitstream. Thechroma_sampling_grid_info_presented_flag being equal to 0 may indicatethat no chroma sampling grid information might be (or in someembodiments perhaps will be) presented. In the latter case, perhaps ifsampling_grid_info_presented_flag is equal to 1, among other reasons,the chroma sampling grid offsets may be derived according to the offsetsfor luma, for example. Perhaps if sampling_grid_info_presented_flag isequal to 0, among other reasons, the chroma sampling grid offsets may beset to the default value 0, for example.

The log 2_chroma_sampling_grid_precision may indicate the precision ofthe chroma sampling grid, as described herein for example. TheChromaSamplingGridPrecision may be determined byChromaSamplingGridPrecision=2^(log 2) ^(_) ^(chroma) ^(_) ^(sampling)^(_) ^(grid) ^(_) ^(precision). The chroma_sampling_grid_abs_offset_xmay indicate the absolute value of the chroma sampling grid offset inthe horizontal direction. The chroma_sampling_grid_abs_offset_x may befixed-lengh coded. The code length may be equal to log2_chroma_sampling_grid_precision. The chroma_sampling_grid_abs_offset_ymay specify the absolute value of the chroma sampling grid offset in thevertical direction. The chroma_sampling_grid_abs_offset_y may befixed-length coded. The code length may be equal to log2_chroma_sampling_grid_precision.

The chroma_sampling_grid_sign_x may indicate the chroma sampling gridoffset in the horizontal direction. The chroma_sampling_grid_sign_xbeing equal to 0 may indicate that the chroma sampling grid offset inthe horizontal direction is positive. The chroma_sampling_grid_sign_xbeing equal to 1 may indicate that the chroma sampling grid offset inthe horizontal direction is negative.

The chroma_sampling_grid_sign_y may indicate the chroma sampling gridoffset in the vertical direction. The chroma_sampling_grid_sign_y beingequal to 0 may indicate that the chroma sampling grid offset in thevertical direction is positive. The chroma_sampling_grid_sign_y beingequal to 1 may indicate that the chroma sampling grid offset in thevertical direction is negative.

The chroma sampling grid offset in the horizontal direction may beindicated by ChromaSamplingGridOffsetX. The ChromaSamplingGridOffsetXmay be determined byChromaSamplingGridOffsetX=(1−chroma_sampling_grid_sign_x*2)*chroma_sampling_grid_abs_offset_x.

The chroma sampling grid offset in the vertical direction may beindicated by ChromaSamplingGridOffsetY. The ChromaSamplingGridOffsetYmay be determined byChromaSamplingGridOffsetY=(1−chroma_sampling_grid_sign_y*2)*chroma_sampling_grid_abs_offset_y.

The syntax and semantics described herein are provided as examples.Alternative, or additional, embodiments may be implemented for codingthe sampling grid information. The variable length coding may be appliedto code the sampling grid offsets. The chroma sampling grid informationmay be directly derived based on the luma sampling grid informationand/or the chorma format information. In some embodiments, one or more,or each, of the chroma_sampling_grid_info_present_flag, log2_chroma_sampling_grid_precision, chroma_sampling_grid_abs_offset_x,chroma_sampling_grid_abs_offset_y, chroma_sampling_grid_sign_x, and/orchroma_sampling_grid_sign_y in Table 2 may be skipped.

In one or more embodiments, the sampling grid informationsampling_grid_information( ) in Table 2 may be signaled as part of theSequence Parameter Set (SPS), as shown in Table 2A for example. Thesampling grid information may be signaled in the Video Parameter Sets(VPS). The VPS may include an extension vps_extension( ) which may beused to specify high level information about the scalable bitstream,such as the layer dependency information between different layers forexample. As shown in Table 2B, the binary flagdirect_dependency_flag[i][j] may be used to indicate whether the currentlayer “i” uses layer “j” as the reference layer for coding. Perhaps whensignaled in VPS, among other scenarios, the relative sampling gridbetween the current layer and one or more reference layers (e.g.,sampling_grid_information(i, j)) may be signaled together with the layerdependency information, for example. The signaling may be performedusing the syntax example illustrated in Table 2B.

In one or more embodiments, perhaps even if layer dependency informationdirect_dependency_flag[i][j] may show that layer “i” depends onreference layer “j” for coding, among other reasons,sampling_grid_information(i, j) may be relevant (and in some embodimentsmay only be relevant) perhaps when spatial scalability may be appliedbetween layer “i” and layer “j”, among other scenarios. In someembodiments, it may be irrelevant perhaps when one or more otherscalabilities, such as view scalability and/or SNR scalability forexample, among other scenarios, may be used. Embodiments recognize thatin the current SHVC design, spatial scalability and/or SNR scalabilitymay not be distinguished in VPS, which may indicate that they may sharethe same scalability mask value, for example a scalability_mask, asshown in Table 3. In some embodiments, Table 3 may be modified andspatial scalability and SNR scalability may have distinct values, forexample as shown in Table 2C. The scalability_mask[i][j] equal to 1 mayindicate that the spatial scalability between the current layer “i” andthe reference layer “j” is applied (or may be applied). In one or moreembodiments, in Table 2C, the element “if (direct_dependency_flag[i][j]),” as shown in Table 2B may be modified as “if(direct_dependency_flag[i] [j] && scalability_mask[i][j]==1),” as shownin Table 2D, such that the sampling grid informationsampling_grid_information( ) may be signaled in VPS, perhaps when (andin some embodiments perhaps only when) the current layer “i” uses layer“j” as the reference layer and/or spatial scalability is applied, amongother reasons, for example.

TABLE 2A Example signaling sampling grid information in SPSsequence_param_set( ) { Descriptor  sps_video_parameter_set_id u(4) ......  sampling_grid_information( )  ...... }

TABLE 2B Example signaling sampling grid information in VPS extensionvps_extension( ) { Descriptor  ......  for( i = 1; i <=vps_max_layers_minus1; i++ )   for( j = 0; j < i; j++ ) {   direct_dependency_flag[ i ][ j ] u(1)     if (direct_dependency_flag[i ] [ j ] )      sampling_grid_information( i , j ) }

TABLE 2C Example mapping of ScalabiltyId to scalability dimensionsscalability_mask Scalability ScalabilityId index dimension mapping 0multiview ViewId 1 Spatial DependencyId scalability 2 SNR scalabilityDependencyId 3-15 Reserved

TABLE 2D Example signaling sampling grid information in VPS extensionvps_extension( ) { Descriptor  ......  for( i = 1; i <=vps_max_layers_minus1; i++ )   for( j = 0; j < i; j++ ) {   direct_dependency_flag[ i ][ j ] u(1)    if (direct_dependency_flag[i ][ j ]&&    scalability_mask[i][j] == 1)     sampling_grid_information( i , j ) }

TABLE 3 Example mapping of ScalabiltyId to scalability dimensionsscalability_mask Scalability ScalabilityId index dimension mapping 0multiview ViewId 1 spatial/SNR DependencyId scalability 2-15 Reserved

In some embodiments, perhaps if the sampling grid information mayindicate that there is non-zero relative phase shift between the twospatial layers, among other scenarios, sampling grid correction may beperformed to align the sampling grid of the up-sampled BL video withthat of the EL video, for example. The corrected signal provided by thealignment may serve as a better prediction for coding of the EL video(e.g., an improved prediction for coding of the EL video as compared tothe prediction before and/or without the alignment). In the example ofFIG. 7, the phase of the shaded pixels may be corrected such that theymay be placed into alignment (and/or be substantially aligned) with theempty squares.

The sampling grid correction may be performed by upsampling and/orfiltering. In one or more embodiments, the BL picture may be upsampledusing the upsampling filters defined in one or more standards (e.g.,normative upsampling). The upsampled BL picture may be filtered usingphase-correction filters. One or more poly-phase filters (or phasecorrection filters) may be used to perform sampling grid correction. Forexample, a poly-phase filter may be configured as one or more of: abilinear filter (e.g., that may correspond to the amount of phaseshift), a bi-cubic filter, a Lanczos filter, and/or an interpolationfilter. Video coding, such as H.264/AVC and/or HEVC video coding forexample, may support fractional pixel motion compensation. Theinterpolation filters that may be used for motion compensation in thesesystems, such as the 6-tap Wiener filter in H.264/AVC or the DCT-IF inHEVC for example, may be used to correct and/or align the samplinggrids. These interpolation filters may be supported by the codingsystem. In some embodiments, one or more of the aforementioned filtersmay be used in one or more combinations. In some embodiments, perhapswhere interpolation filters may be used for motion compensation, amongother scenarios, additional filtering functionalities might not beimplemented.

The sampling grid correction may be performed using the sampling gridinformation described herein. In some embodiments, the upsamplingfiltering process defined in the scalable extension of HEVC (SHVC) maybe used. The sampling grid correction may be performed using 1.5× or 2×spatial scalability and/or 4:2:0 color subsampling format.

Using the upsampling ratio N in the horizontal and vertical directions,the upsampling process may be approximated by interpolating the BLpicture to 16 times its size in one or both directions and/or decimatingthe 16× picture with the ratio M, where 16≈M×N. To perform 16×upsampling, among other reasons, 16-phase interpolation filters may beused. Examples of detailed filter coefficients are provided in Table 4and Table 5 for luma and for chroma, respectively. Embodiments recognizethat SHVC may support 1.5× and/or 2× spatial scalability. A subset ofthe 16-phase interpolation filters may be described in the scalabilityextension of HEVC (SHVC) (e.g., as shown in the rows of Table 4 at phasep=0, 5, 8, and 11 and/or in the rows of Table 5 at phase p=0, 4, 5, 6,8, 9, 11, 14, and 15). Embodiments contemplate that the phase filtersthat might not be used in the 2× or the 1.5× upsampling process mightnot be described by SHVC. In one or more embodiments, the luma filtersmay be 8-tap. In one or more embodiments, the chroma filters may be4-tap. Also, in one or more embodiments, the luma filters and/or chromafilters may be 2D separable filters.

TABLE 4 Example 16-phase luma upsampling filter phase filtercoefficients p f_(L)[p, 0] f_(L)[p, 1] f_(L)[p, 2] f_(L)[p, 3] f_(L)[p,4] f_(L)[p, 5] f_(L)[p, 6] f_(L)[p, 7] 0 0 0 0 64 0 0 0 0 1 0 1 −3 63 4−2 1 0 2 0 2 −6 61 9 −3 1 0 3 −1 3 −8 60 13 −4 1 0 4 −1 4 −10 58 17 −5 10 5 −1 4 −11 52 26 −8 3 −1 6 −1 4 −11 50 29 −9 3 −1 7 −1 4 −11 45 34 −104 −1 8 −1 4 −11 40 40 −11 4 −1 9 −1 4 −10 34 45 −11 4 −1 10 −1 3 −9 2950 −11 4 −1 11 −1 3 −8 26 52 −11 4 −1 12 0 1 −5 17 58 −10 4 −1 13 0 1 −413 60 −8 3 −1 14 0 1 −3 9 61 −6 2 0 15 0 1 −2 4 63 −3 1 0

TABLE 5 Example 16-phase chroma upsampling filter filter coefficientsphase p f_(C)[p, 0] f_(C)[p, 1] f_(C)[p, 2] f_(C)[p, 3] 0 0 64 0 0 1 −262 4 0 2 −2 58 10 −2 3 −4 56 14 −2 4 −4 54 16 −2 5 −6 52 20 −2 6 −6 4628 −4 7 −4 42 30 −4 8 −4 36 36 −4 9 −4 30 42 −4 10 −4 28 46 −6 11 −2 2052 −6 12 −2 16 54 −4 13 −2 14 56 −4 14 −2 10 58 −2 15 0 4 62 −2

The 16× interpolation may or might not be performed for one or more, oreach, sample. In some embodiments, the samples in the 16× picture thatmay be kept after decimation may be interpolated. To generate a samplelocated as (x, y) in the EL picture, among other reasons, thecorresponding position (x16, y16) in the virtual 16× picture beforedecimation may be found by using Equation (3) and/or Equation (4).x16=(x*BLPictureWidth*16+ELPictureWidth/2)/ELPictureWidth  Equation (3)y16=(y*BLPictureHeight*16+ELPictureHeight/2)/ELPictureHeight−offset  Equation(4)The (BLPictureWidth, BLPictureHeight) and (ELPictureWidth,ELPictureHeight) may represent picture dimensions of BL and EL pictures,respectively. The dimensions may be of luma or chroma planes, accordingto which planes the (x16, y16) may be derived for. In Equation (4),offset may be defined as described herein. Perhaps if y16 is calculatedfor a luma plane, among other scenarios, the offset may be equal to 0.Perhaps if y16 is calculated for a chroma plane and/or the ratio forspatial scalability is be 1.5×, among other scenarios, the offset may beequal to 1. Perhaps if y16 is calculated for a chroma plane and/or theratio for spatial scalability is 2×, among other scenarios, the offsetmay be equal to 2.

The sample value at (x16, y16) may be generated by applying theappropriate phase filter to the support region from the BL picture. Theindex of phase filter for the horizontal direction may be calculated as(x16%16). The index of phase filter for the vertical direction may becalculated as (y16%16). The position of the pixel in BL picture wherethe phase filter may be anchored may be calculated as (floor (x16/16),floor(y16/16)). The floor(.) function may be used to map a real numberto the largest previous integer. In Equation (3) and/or Equation (4),the derivation of (x16, y16) may assume the sampling grids of BL and ELpictures have zero phase shift. The (x16, y16) may be calculated usingthe sampling grid information signaled in the sequence header, such asVPS and/or SPS for example, and/or using BL and/or EL picture dimensioninformation.

For a luma plane, (x16, y16) may be derived as shown in Equation (5)and/or Equation (6).x16=(x*BLPictureWidth*16+ELPictureWidth/2)/ELPictureWidth−SamplingGridOffsetX  Equation(5)y16=(y*BLPictureHeight*16+ELPictureHeight/2)/ELPictureHeight−offset−SamplingGridOffsetY  Equation(6)

For chroma planes, perhaps if sampling grid offsets are presentedexplicitly (e.g., chroma_sampling_grid_info_presented_flag is equal to1), among other reasons, (x16, y16) may be derived as shown in Equation(7) and/or Equation (8).x16=(x*BLPictureWidth*16+ELPictureWidth/2)/ELPictureWidth−ChromaSamplingGridOffsetX  Equation(7)y16=(y*BLPictureHeight*16)/ELPictureHeight−offset−ChromaSamplingGridOffsetY  Equation(8)

For chroma planes, perhaps if sampling grid offsets are not presentedexplicitly (e.g., chroma_sampling_grid_info_presented_flag is signaledand set to zero, or is not signaled and/or determined to be zero), amongother reasons, (x16, y16) may be derived according to the sampling gridoffsets for luma, as shown in Equation (9) and/or Equation (10).x16=(x*BLPictureWidth*16+ELPictureWidth/2)/ELPictureWidth−SamplingGridOffsetX/2  Equation(9)y16=(y*BLPictureHeight*16)/ELPictureHeight−offset−SamplingGridOffsetY/2  Equation(10)

The sampling grids may be detected, as described herein for example. Thedown-sampling process, as shown in FIG. 3 for example, may or might notbe a part of the encoder. In some applications, the encoder may receivethe higher resolution EL video, perform down-sampling to obtain thelower resolution BL video, and/or apply spatial scalable encoding toboth video inputs. The encoder may compare the down-sampling filterphase shift characteristics with the up-sampling filter phase shiftcharacteristics to determine the relative phase shift value, among otherreasons. The encoder may directly receive the higher resolution EL videoand/or the lower resolution BL video as inputs, for example, perhapsafter the down-sampling process has been applied elsewhere in thepipeline, among other scenarios. The phase shift characteristics of thedown-sampling filters may be unknown to the encoder and/or the encodermay detect the relative sampling grids.

FIG. 10 shows an example for using adaptive filter training to detectthe relative sampling grids. The BL video may be up-sampled using theup-sampling filter to match the EL video spatial resolution. Adaptivefilter training (e.g., a Least Squared linear filter training) may beapplied on the EL input and the up-sampled BL input. In someembodiments, the relative phase shift may be detected by examining theone or more adaptive filter coefficients. The adaptive filtercoefficients (e.g., those obtained using LS training) may be of floatingpoint precision. In some embodiments, coefficient quantization of atleast one of the one or more adaptive filter coefficients may be used todetermine where the relative sampling grid position may be according toa predefined fixed precision (e.g., as in FIG. 8). Adaptive filtertraining may be performed in one or more, or each, dimension separatelyto detect relative sampling grid in the given dimension, for example,among other reasons.

Using the X dimension as an example, FIG. 11 shows an example of trainedcoefficients of an adaptive filter of tap length 2 and the relativesampling grids between the up-sampled BL (patterned squares) and the EL(empty squares) it may correspond to. As described herein, the relativesampling grid information may be fixed throughout the duration of theinput video. Detection of the sampling grids may be performed (e.g., atleast once) at the beginning of the input video, for example.

FIG. 12 shows an enhanced 2-layer scalable encoder (e.g., as shown inFIG. 3) that may incorporate one or more of the embodiments describedherein, such as sampling grid information signaling, sampling gridcorrection, and/or sampling grid detection. Though not shown, thescalable decoder illustrated in FIG. 4 may be implemented with one ormore of these embodiments in substantially similar manners. Whileembodiments are described herein with an example 2-layer spatialscalable system, these embodiments may be applicable to spatial scalablesystems that support more layers. For example, the embodiments describedherein may be applicable to any two spatial layers in a multi-layersystem. The layers in an N-layer scalable system may include (L₀, L₁ . .. L_(N−1)), and it may be assumed that (L_(a), L_(b)) and (L_(a),L_(d)), 0≤a, b, c, d≤(N−1), may be two pairs of spatial layers withdifferent spatial scaling ratios and different relative phase shiftcharacteristics. For example, (L_(a), L_(b)) may have 1.5× ratio withone-half pixel phase shift. (L_(a), L_(d)) may have 2× ratio withone-fourth pixel phase shift. The signaling illustrated in FIG. 8 may beexpanded to include layer indications, which may be followed by samplinggrid information signaling for the particular layers. Since layerdependency information may be signaled in VPS, compared to SPS forexample, VPS may be a place to include the layer-specific sampling gridinformation.

FIG. 13A is a diagram of an example communications system 100 in whichone or more disclosed embodiments may be implemented. The communicationssystem 100 may be a multiple access system that provides content, suchas voice, data, video, messaging, broadcast, etc., to multiple wirelessusers. The communications system 100 may enable multiple wireless usersto access such content through the sharing of system resources,including wireless bandwidth. For example, the communications systems100 may employ one or more channel access methods, such as code divisionmultiple access (CDMA), time division multiple access (TDMA), frequencydivision multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrierFDMA (SC-FDMA), and the like.

As shown in FIG. 13A, the communications system 100 may include wirelesstransmit/receive units (WTRUs) 102 a, 102 b, 102 c, and/or 102 d (whichgenerally or collectively may be referred to as WTRU 102), a radioaccess network (RAN) 103/104/105, a core network 106/107/109, a publicswitched telephone network (PSTN) 108, the Internet 110, and othernetworks 112, though it will be appreciated that the disclosedembodiments contemplate any number of WTRUs, base stations, networks,and/or network elements. Each of the WTRUs 102 a, 102 b, 102 c, 102 dmay be any type of device configured to operate and/or communicate in awireless environment. By way of example, the WTRUs 102 a, 102 b, 102 c,102 d may be configured to transmit and/or receive wireless signals andmay include user equipment (UE), a mobile station, a fixed or mobilesubscriber unit, a pager, a cellular telephone, a personal digitalassistant (PDA), a smartphone, a laptop, a netbook, a personal computer,a wireless sensor, consumer electronics, and the like.

The communications systems 100 may also include a base station 114 a anda base station 114 b. Each of the base stations 114 a, 114 b may be anytype of device configured to wirelessly interface with at least one ofthe WTRUs 102 a, 102 b, 102 c, 102 d to facilitate access to one or morecommunication networks, such as the core network 106/107/109, theInternet 110, and/or the networks 112. By way of example, the basestations 114 a, 114 b may be a base transceiver station (BTS), a Node-B,an eNode B, a Home Node B, a Home eNode B, a site controller, an accesspoint (AP), a wireless router, and the like. While the base stations 114a, 114 b are each depicted as a single element, it will be appreciatedthat the base stations 114 a, 114 b may include any number ofinterconnected base stations and/or network elements.

The base station 114 a may be part of the RAN 103/104/105, which mayalso include other base stations and/or network elements (not shown),such as a base station controller (BSC), a radio network controller(RNC), relay nodes, etc. The base station 114 a and/or the base station114 b may be configured to transmit and/or receive wireless signalswithin a particular geographic region, which may be referred to as acell (not shown). The cell may further be divided into cell sectors. Forexample, the cell associated with the base station 114 a may be dividedinto three sectors. Thus, in one embodiment, the base station 114 a mayinclude three transceivers, i.e., one for each sector of the cell. Inanother embodiment, the base station 114 a may employ multiple-inputmultiple output (MIMO) technology and, therefore, may utilize multipletransceivers for each sector of the cell.

The base stations 114 a, 114 b may communicate with one or more of theWTRUs 102 a, 102 b, 102 c, 102 d over an air interface 115/116/117,which may be any suitable wireless communication link (e.g., radiofrequency (RF), microwave, infrared (IR), ultraviolet (UV), visiblelight, etc.). The air interface 115/116/117 may be established using anysuitable radio access technology (RAT).

More specifically, as noted above, the communications system 100 may bea multiple access system and may employ one or more channel accessschemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. Forexample, the base station 114 a in the RAN 103/104/105 and the WTRUs 102a, 102 b, 102 c may implement a radio technology such as UniversalMobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA),which may establish the air interface 115/116/117 using wideband CDMA(WCDMA). WCDMA may include communication protocols such as High-SpeedPacket Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may includeHigh-Speed Downlink Packet Access (HSDPA) and/or High-Speed UplinkPacket Access (HSUPA).

In another embodiment, the base station 114 a and the WTRUs 102 a, 102b, 102 c may implement a radio technology such as Evolved UMTSTerrestrial Radio Access (E-UTRA), which may establish the air interface115/116/117 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A).

In other embodiments, the base station 114 a and the WTRUs 102 a, 102 b,102 c may implement radio technologies such as IEEE 802.16 (i.e.,Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000,CDMA2000 1×, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), InterimStandard 95 (IS-95), Interim Standard 856 (IS-856), Global System forMobile communications (GSM), Enhanced Data rates for GSM Evolution(EDGE), GSM EDGE (GERAN), and the like.

The base station 114 b in FIG. 13A may be a wireless router, Home NodeB, Home eNode B, or access point, for example, and may utilize anysuitable RAT for facilitating wireless connectivity in a localized area,such as a place of business, a home, a vehicle, a campus, and the like.In one embodiment, the base station 114 b and the WTRUs 102 c, 102 d mayimplement a radio technology such as IEEE 802.11 to establish a wirelesslocal area network (WLAN). In another embodiment, the base station 114 band the WTRUs 102 c, 102 d may implement a radio technology such as IEEE802.15 to establish a wireless personal area network (WPAN). In yetanother embodiment, the base station 114 b and the WTRUs 102 c, 102 dmay utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE,LTE-A, etc.) to establish a picocell or femtocell. As shown in FIG. 13A,the base station 114 b may have a direct connection to the Internet 110.Thus, the base station 114 b may not be required to access the Internet110 via the core network 106/107/109.

The RAN 103/104/105 may be in communication with the core network106/107/109, which may be any type of network configured to providevoice, data, applications, and/or voice over internet protocol (VoIP)services to one or more of the WTRUs 102 a, 102 b, 102 c, 102 d. Forexample, the core network 106/107/109 may provide call control, billingservices, mobile location-based services, pre-paid calling, Internetconnectivity, video distribution, etc., and/or perform high-levelsecurity functions, such as user authentication. Although not shown inFIG. 13A, it will be appreciated that the RAN 103/104/105 and/or thecore network 106/107/109 may be in direct or indirect communication withother RANs that employ the same RAT as the RAN 103/104/105 or adifferent RAT. For example, in addition to being connected to the RAN103/104/105, which may be utilizing an E-UTRA radio technology, the corenetwork 106/107/109 may also be in communication with another RAN (notshown) employing a GSM radio technology.

The core network 106/107/109 may also serve as a gateway for the WTRUs102 a, 102 b, 102 c, 102 d to access the PSTN 108, the Internet 110,and/or other networks 112. The PSTN 108 may include circuit-switchedtelephone networks that provide plain old telephone service (POTS). TheInternet 110 may include a global system of interconnected computernetworks and devices that use common communication protocols, such asthe transmission control protocol (TCP), user datagram protocol (UDP)and the internet protocol (IP) in the TCP/IP internet protocol suite.The networks 112 may include wired or wireless communications networksowned and/or operated by other service providers. For example, thenetworks 112 may include another core network connected to one or moreRANs, which may employ the same RAT as the RAN 103/104/105 or adifferent RAT.

Some or all of the WTRUs 102 a, 102 b, 102 c, 102 d in thecommunications system 100 may include multi-mode capabilities, i.e., theWTRUs 102 a, 102 b, 102 c, 102 d may include multiple transceivers forcommunicating with different wireless networks over different wirelesslinks. For example, the WTRU 102 c shown in FIG. 13A may be configuredto communicate with the base station 114 a, which may employ acellular-based radio technology, and with the base station 114 b, whichmay employ an IEEE 802 radio technology.

FIG. 13B is a system diagram of an example WTRU 102. As shown in FIG.13B, the WTRU 102 may include a processor 118, a transceiver 120, atransmit/receive element 122, a speaker/microphone 124, a keypad 126, adisplay/touchpad 128, non-removable memory 130, removable memory 132, apower source 134, a global positioning system (GPS) chipset 136, andother peripherals 138. It will be appreciated that the WTRU 102 mayinclude any sub-combination of the foregoing elements while remainingconsistent with an embodiment. Also, embodiments contemplate that thebase stations 114 a and 114 b, and/or the nodes that base stations 114 aand 114 b may represent, such as but not limited to transceiver station(BTS), a Node-B, a site controller, an access point (AP), a home node-B,an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a homeevolved node-B gateway, and proxy nodes, among others, may include someor all of the elements depicted in FIG. 13B and described herein.

The processor 118 may be a general purpose processor, a special purposeprocessor, a conventional processor, a digital signal processor (DSP), aplurality of microprocessors, one or more microprocessors in associationwith a DSP core, a controller, a microcontroller, Application SpecificIntegrated Circuits (ASICs), Field Programmable Gate Array (FPGAs)circuits, any other type of integrated circuit (IC), a state machine,and the like. The processor 118 may perform signal coding, dataprocessing, power control, input/output processing, and/or any otherfunctionality that enables the WTRU 102 to operate in a wirelessenvironment. The processor 118 may be coupled to the transceiver 120,which may be coupled to the transmit/receive element 122. While FIG. 13Bdepicts the processor 118 and the transceiver 120 as separatecomponents, it will be appreciated that the processor 118 and thetransceiver 120 may be integrated together in an electronic package orchip.

The transmit/receive element 122 may be configured to transmit signalsto, or receive signals from, a base station (e.g., the base station 114a) over the air interface 115/116/117. For example, in one embodiment,the transmit/receive element 122 may be an antenna configured totransmit and/or receive RF signals. In another embodiment, thetransmit/receive element 122 may be an emitter/detector configured totransmit and/or receive IR, UV, or visible light signals, for example.In yet another embodiment, the transmit/receive element 122 may beconfigured to transmit and receive both RF and light signals. It will beappreciated that the transmit/receive element 122 may be configured totransmit and/or receive any combination of wireless signals.

In addition, although the transmit/receive element 122 is depicted inFIG. 13B as a single element, the WTRU 102 may include any number oftransmit/receive elements 122. More specifically, the WTRU 102 mayemploy MIMO technology. Thus, in one embodiment, the WTRU 102 mayinclude two or more transmit/receive elements 122 (e.g., multipleantennas) for transmitting and receiving wireless signals over the airinterface 115/116/117.

The transceiver 120 may be configured to modulate the signals that areto be transmitted by the transmit/receive element 122 and to demodulatethe signals that are received by the transmit/receive element 122. Asnoted above, the WTRU 102 may have multi-mode capabilities. Thus, thetransceiver 120 may include multiple transceivers for enabling the WTRU102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, forexample.

The processor 118 of the WTRU 102 may be coupled to, and may receiveuser input data from, the speaker/microphone 124, the keypad 126, and/orthe display/touchpad 128 (e.g., a liquid crystal display (LCD) displayunit or organic light-emitting diode (OLED) display unit). The processor118 may also output user data to the speaker/microphone 124, the keypad126, and/or the display/touchpad 128. In addition, the processor 118 mayaccess information from, and store data in, any type of suitable memory,such as the non-removable memory 130 and/or the removable memory 132.The non-removable memory 130 may include random-access memory (RAM),read-only memory (ROM), a hard disk, or any other type of memory storagedevice. The removable memory 132 may include a subscriber identitymodule (SIM) card, a memory stick, a secure digital (SD) memory card,and the like. In other embodiments, the processor 118 may accessinformation from, and store data in, memory that is not physicallylocated on the WTRU 102, such as on a server or a home computer (notshown).

The processor 118 may receive power from the power source 134, and maybe configured to distribute and/or control the power to the othercomponents in the WTRU 102. The power source 134 may be any suitabledevice for powering the WTRU 102. For example, the power source 134 mayinclude one or more dry cell batteries (e.g., nickel-cadmium (NiCd),nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion),etc.), solar cells, fuel cells, and the like.

The processor 118 may also be coupled to the GPS chipset 136, which maybe configured to provide location information (e.g., longitude andlatitude) regarding the current location of the WTRU 102. In additionto, or in lieu of, the information from the GPS chipset 136, the WTRU102 may receive location information over the air interface 115/116/117from a base station (e.g., base stations 114 a, 114 b) and/or determineits location based on the timing of the signals being received from twoor more nearby base stations. It will be appreciated that the WTRU 102may acquire location information by way of any suitablelocation-determination method while remaining consistent with anembodiment.

The processor 118 may further be coupled to other peripherals 138, whichmay include one or more software and/or hardware modules that provideadditional features, functionality and/or wired or wirelessconnectivity. For example, the peripherals 138 may include anaccelerometer, an e-compass, a satellite transceiver, a digital camera(for photographs or video), a universal serial bus (USB) port, avibration device, a television transceiver, a hands free headset, aBluetooth® module, a frequency modulated (FM) radio unit, a digitalmusic player, a media player, a video game player module, an Internetbrowser, and the like.

FIG. 13C is a system diagram of the RAN 103 and the core network 106according to an embodiment. As described herein, the RAN 103 may employa UTRA radio technology to communicate with the WTRUs 102 a, 102 b, 102c over the air interface 115. The RAN 103 may also be in communicationwith the core network 106. As shown in FIG. 13C, the RAN 103 may includeNode-Bs 140 a, 140 b, 140 c, which may each include one or moretransceivers for communicating with the WTRUs 102 a, 102 b, 102 c overthe air interface 115. The Node-Bs 140 a, 140 b, 140 c may each beassociated with a particular cell (not shown) within the RAN 103. TheRAN 103 may also include RNCs 142 a, 142 b. It will be appreciated thatthe RAN 103 may include any number of Node-Bs and RNCs while remainingconsistent with an embodiment.

As shown in FIG. 13C, the Node-Bs 140 a, 140 b may be in communicationwith the RNC 142 a. Additionally, the Node-B 140 c may be incommunication with the RNC 142 b. The Node-Bs 140 a, 140 b, 140 c maycommunicate with the respective RNCs 142 a, 142 b via an Iub interface.The RNCs 142 a, 142 b may be in communication with one another via anIur interface. Each of the RNCs 142 a, 142 b may be configured tocontrol the respective Node-Bs 140 a, 140 b, 140 c to which it isconnected. In addition, each of the RNCs 142 a, 142 b may be configuredto carry out or support other functionality, such as outer loop powercontrol, load control, admission control, packet scheduling, handovercontrol, macrodiversity, security functions, data encryption, and thelike.

The core network 106 shown in FIG. 13C may include a media gateway (MGW)144, a mobile switching center (MSC) 146, a serving GPRS support node(SGSN) 148, and/or a gateway GPRS support node (GGSN) 150. While each ofthe foregoing elements are depicted as part of the core network 106, itwill be appreciated that any one of these elements may be owned and/oroperated by an entity other than the core network operator.

The RNC 142 a in the RAN 103 may be connected to the MSC 146 in the corenetwork 106 via an IuCS interface. The MSC 146 may be connected to theMGW 144. The MSC 146 and the MGW 144 may provide the WTRUs 102 a, 102 b,102 c with access to circuit-switched networks, such as the PSTN 108, tofacilitate communications between the WTRUs 102 a, 102 b, 102 c andtraditional land-line communications devices.

The RNC 142 a in the RAN 103 may also be connected to the SGSN 148 inthe core network 106 via an IuPS interface. The SGSN 148 may beconnected to the GGSN 150. The SGSN 148 and the GGSN 150 may provide theWTRUs 102 a, 102 b, 102 c with access to packet-switched networks, suchas the Internet 110, to facilitate communications between and the WTRUs102 a, 102 b, 102 c and IP-enabled devices.

As noted above, the core network 106 may also be connected to thenetworks 112, which may include other wired or wireless networks thatare owned and/or operated by other service providers.

FIG. 13D is a system diagram of the RAN 104 and the core network 107according to an embodiment. As noted above, the RAN 104 may employ anE-UTRA radio technology to communicate with the WTRUs 102 a, 102 b, 102c over the air interface 116. The RAN 104 may also be in communicationwith the core network 107.

The RAN 104 may include eNode-Bs 160 a, 160 b, 160 c, though it will beappreciated that the RAN 104 may include any number of eNode-Bs whileremaining consistent with an embodiment. The eNode-Bs 160 a, 160 b, 160c may each include one or more transceivers for communicating with theWTRUs 102 a, 102 b, 102 c over the air interface 116. In one embodiment,the eNode-Bs 160 a, 160 b, 160 c may implement MIMO technology. Thus,the eNode-B 160 a, for example, may use multiple antennas to transmitwireless signals to, and receive wireless signals from, the WTRU 102 a.

Each of the eNode-Bs 160 a, 160 b, 160 c may be associated with aparticular cell (not shown) and may be configured to handle radioresource management decisions, handover decisions, scheduling of usersin the uplink and/or downlink, and the like. As shown in FIG. 13D, theeNode-Bs 160 a, 160 b, 160 c may communicate with one another over an X2interface.

The core network 107 shown in FIG. 13D may include a mobility managementgateway (MME) 162, a serving gateway 164, and a packet data network(PDN) gateway 166. While each of the foregoing elements are depicted aspart of the core network 107, it will be appreciated that any one ofthese elements may be owned and/or operated by an entity other than thecore network operator.

The MME 162 may be connected to each of the eNode-Bs 160 a, 160 b, 160 cin the RAN 104 via an S1 interface and may serve as a control node. Forexample, the MME 162 may be responsible for authenticating users of theWTRUs 102 a, 102 b, 102 c, bearer activation/deactivation, selecting aparticular serving gateway during an initial attach of the WTRUs 102 a,102 b, 102 c, and the like. The MME 162 may also provide a control planefunction for switching between the RAN 104 and other RANs (not shown)that employ other radio technologies, such as GSM or WCDMA.

The serving gateway 164 may be connected to each of the eNode-Bs 160 a,160 b, 160 c in the RAN 104 via the S1 interface. The serving gateway164 may generally route and forward user data packets to/from the WTRUs102 a, 102 b, 102 c. The serving gateway 164 may also perform otherfunctions, such as anchoring user planes during inter-eNode B handovers,triggering paging when downlink data is available for the WTRUs 102 a,102 b, 102 c, managing and storing contexts of the WTRUs 102 a, 102 b,102 c, and the like.

The serving gateway 164 may also be connected to the PDN gateway 166,which may provide the WTRUs 102 a, 102 b, 102 c with access topacket-switched networks, such as the Internet 110, to facilitatecommunications between the WTRUs 102 a, 102 b, 102 c and IP-enableddevices.

The core network 107 may facilitate communications with other networks.For example, the core network 107 may provide the WTRUs 102 a, 102 b,102 c with access to circuit-switched networks, such as the PSTN 108, tofacilitate communications between the WTRUs 102 a, 102 b, 102 c andtraditional land-line communications devices. For example, the corenetwork 107 may include, or may communicate with, an IP gateway (e.g.,an IP multimedia subsystem (IMS) server) that serves as an interfacebetween the core network 107 and the PSTN 108. In addition, the corenetwork 107 may provide the WTRUs 102 a, 102 b, 102 c with access to thenetworks 112, which may include other wired or wireless networks thatare owned and/or operated by other service providers.

FIG. 13E is a system diagram of the RAN 105 and the core network 109according to an embodiment. The RAN 105 may be an access service network(ASN) that employs IEEE 802.16 radio technology to communicate with theWTRUs 102 a, 102 b, 102 c over the air interface 117. As will be furtherdiscussed below, the communication links between the differentfunctional entities of the WTRUs 102 a, 102 b, 102 c, the RAN 105, andthe core network 109 may be defined as reference points.

As shown in FIG. 13E, the RAN 105 may include base stations 180 a, 180b, 180 c, and an ASN gateway 182, though it will be appreciated that theRAN 105 may include any number of base stations and ASN gateways whileremaining consistent with an embodiment. The base stations 180 a, 180 b,180 c may each be associated with a particular cell (not shown) in theRAN 105 and may each include one or more transceivers for communicatingwith the WTRUs 102 a, 102 b, 102 c over the air interface 117. In oneembodiment, the base stations 180 a, 180 b, 180 c may implement MIMOtechnology. Thus, the base station 180 a, for example, may use multipleantennas to transmit wireless signals to, and receive wireless signalsfrom, the WTRU 102 a. The base stations 180 a, 180 b, 180 c may alsoprovide mobility management functions, such as handoff triggering,tunnel establishment, radio resource management, traffic classification,quality of service (QoS) policy enforcement, and the like. The ASNgateway 182 may serve as a traffic aggregation point and may beresponsible for paging, caching of subscriber profiles, routing to thecore network 109, and the like.

The air interface 117 between the WTRUs 102 a, 102 b, 102 c and the RAN105 may be defined as an R1 reference point that implements the IEEE802.16 specification. In addition, each of the WTRUs 102 a, 102 b, 102 cmay establish a logical interface (not shown) with the core network 109.The logical interface between the WTRUs 102 a, 102 b, 102 c and the corenetwork 109 may be defined as an R2 reference point, which may be usedfor authentication, authorization, IP host configuration management,and/or mobility management.

The communication link between each of the base stations 180 a, 180 b,180 c may be defined as an R8 reference point that includes protocolsfor facilitating WTRU handovers and the transfer of data between basestations. The communication link between the base stations 180 a, 180 b,180 c and the ASN gateway 182 may be defined as an R6 reference point.The R6 reference point may include protocols for facilitating mobilitymanagement based on mobility events associated with each of the WTRUs102 a, 102 b, 102 c.

As shown in FIG. 13E, the RAN 105 may be connected to the core network109. The communication link between the RAN 105 and the core network 109may defined as an R3 reference point that includes protocols forfacilitating data transfer and mobility management capabilities, forexample. The core network 109 may include a mobile IP home agent(MIP-HA) 184, an authentication, authorization, accounting (AAA) server186, and a gateway 188. While each of the foregoing elements aredepicted as part of the core network 109, it will be appreciated thatany one of these elements may be owned and/or operated by an entityother than the core network operator.

The MIP-HA may be responsible for IP address management, and may enablethe WTRUs 102 a, 102 b, 102 c to roam between different ASNs and/ordifferent core networks. The MIP-HA 184 may provide the WTRUs 102 a, 102b, 102 c with access to packet-switched networks, such as the Internet110, to facilitate communications between the WTRUs 102 a, 102 b, 102 cand IP-enabled devices. The AAA server 186 may be responsible for userauthentication and for supporting user services. The gateway 188 mayfacilitate interworking with other networks. For example, the gateway188 may provide the WTRUs 102 a, 102 b, 102 c with access tocircuit-switched networks, such as the PSTN 108, to facilitatecommunications between the WTRUs 102 a, 102 b, 102 c and traditionalland-line communications devices. In addition, the gateway 188 mayprovide the WTRUs 102 a, 102 b, 102 c with access to the networks 112,which may include other wired or wireless networks that are owned and/oroperated by other service providers.

Although not shown in FIG. 13E, it will be appreciated that the RAN 105may be connected to other ASNs and the core network 109 may be connectedto other core networks. The communication link between the RAN 105 theother ASNs may be defined as an R4 reference point, which may includeprotocols for coordinating the mobility of the WTRUs 102 a, 102 b, 102 cbetween the RAN 105 and the other ASNs. The communication link betweenthe core network 109 and the other core networks may be defined as an R5reference, which may include protocols for facilitating interworkingbetween home core networks and visited core networks.

The above-referenced communications systems may be used to implement oneor more of the embodiments described herein. For example, thecommunications systems, or portions thereof, may be used to transmitand/or align video data as described herein.

Although features and elements are described above in particularcombinations, each feature or element can be used alone or in anycombination with the other features and elements. Additionally, whilefeatures and elements are described in a particular order, thesefeatures and elements are not limited to the order described. Further,the methods described herein may be implemented in a computer program,software, or firmware incorporated in a computer-readable medium forexecution by a computer or processor. Examples of computer-readablemedia include electronic signals (transmitted over wired or wirelessconnections) and computer-readable storage media. Examples ofcomputer-readable storage media include, but are not limited to, a readonly memory (ROM), a random access memory (RAM), a register, cachememory, semiconductor memory devices, magnetic media such as internalhard disks and removable disks, magneto-optical media, and optical mediasuch as CD-ROM disks, and digital versatile disks (DVDs). A processor inassociation with software may be used to implement a radio frequencytransceiver for use in a WTRU, WTRU, terminal, base station, RNC, or anyhost computer.

What is claimed is:
 1. A method for multi-layer video coding, the methodcomprising: receiving a signal including sampling grid information, thesampling grid information including luma plane phase shift data andchroma plane phase shift data, the luma plane phase shift dataspecifying a luma phase shift between a luma sampling grid of a firstpicture in a first video layer and a luma sampling grid of a secondpicture in a second video layer, the chroma plane phase shift dataspecifying a chroma phase shift between a chroma sampling grid of thefirst picture and a chroma sampling grid of the second picture;upsampling the first picture; performing sampling grid correction basedon the sampling grid information during the upsampling of the firstpicture; selecting one or more High Efficiency Video Coding (HEVC)poly-phase filters based on the chroma plane phase shift data and theluma plane phase shift data, the sampling grid correction includingapplying the one or more HEVC poly-phase filters; aligning the lumasampling grid of the first picture with the luma sampling grid of thesecond picture based on the sampling grid correction; and aligning thechroma sampling grid of the first picture with the chroma sampling gridof the second picture based on the sampling grid correction.
 2. Themethod of claim 1, wherein the chroma plane phase shift data includes atleast one of: a horizontal chroma phase shift or a vertical chroma phaseshift.
 3. The method of claim 1, wherein the luma plane phase shift dataincludes at least one of: a horizontal luma phase shift or a verticalluma phase shift.
 4. The method of claim 1, wherein the signal includesa flag, the flag indicating that the sampling grid information isincluded in the signal.
 5. The method of claim 1, wherein the firstvideo layer is a base layer and the second video layer is an enhancementlayer.
 6. The method of claim 1, wherein the chroma plane phase shiftdata and the luma plane phase shift data indicates a non-zero relativephase shift between the first picture in the first video layer and thesecond picture in the second video layer.
 7. A device comprising: areceiver configured to receive a signal including sampling gridinformation, the sampling grid information including luma plane phaseshift data and chroma plane phase shift data, the luma plane phase shiftdata specifying a luma phase shift between a luma sampling grid of afirst picture in a first video layer and a luma sampling grid of asecond picture in a second video layer, the chroma plane phase shiftdata specifying a chroma phase shift between a chroma sampling grid ofthe first picture and a chroma sampling grid of the second picture; anda processor, configured to: upsample the first picture; perform samplinggrid correction based on the sampling grid information during theupsampling of the first picture; select one or more High EfficiencyVideo Coding (HEVC) poly-phase filters based on the chroma plane phaseshift data and the luma plane phase shift data, the processor configuredto apply the one or more HEVC poly-phase filters as part of the samplinggrid correction; align the luma sampling grid of the first picture withthe luma sampling grid of the second picture based on the sampling gridcorrection; and align the chroma sampling grid of the first picture withthe chroma sampling grid of the second picture based on the samplinggrid correction.
 8. The device of claim 7, wherein the chroma planephase shift data includes at least one of: a horizontal chroma phaseshift or a vertical chroma phase shift.
 9. The device of claim 7,wherein the luma plane phase shift data includes at least one of: ahorizontal luma phase shift or a vertical luma phase shift.
 10. Thedevice of claim 7, wherein the signal includes a flag, the flagindicating that the sampling grid information is included in the signal.11. The device of claim 7, wherein the chroma plane phase shift data andthe luma plane phase shift data indicates a non-zero relative phaseshift between the first picture in the first video layer and the secondpicture in the second video layer.
 12. A method for multi-layer videocoding, the method comprising: receiving a signal including samplinggrid information, the sampling grid information including luma planephase shift data and chroma plane phase shift data, the luma plane phaseshift data specifying a luma phase shift between a base layer (BL)picture and an enhancement layer (EL) picture, the chroma plane phaseshift data specifying a chroma phase shift between the BL picture andthe EL picture; selecting one or more High Efficiency Video Coding(HEVC) poly-phase filters based on the chroma plane phase shift data andthe luma plane phase shift data; and upsampling the BL picture byapplying the one or more HEVC polyphase filters to the BL picture, theapplication of the one or more HEVC polyphase filters comprising:aligning a luma sampling grid of the upsampled BL picture with a lumasampling grid of the EL picture; and aligning a chroma sampling grid ofthe upsampled BL picture with a chroma sampling grid of the EL picture.13. The method of claim 12, wherein the chroma plane phase shift dataand the luma plane phase shift data indicate non-zero relative phaseshifts in chroma and luma respectively between the BL picture and the ELpicture.